The present invention relates to image predictive decoding and image predictive coding and, more particularly, to image predictive decoding methods, image predictive decoding apparatuses, image predictive coding methods, image predictive coding apparatuses, and data storage media, which are used for processing variable-size images.
In order to store or transmit a digital image with high efficiency, it is necessary to compressively code the digital image. As a typical method for compressive coding of a digital image, there is DCT (Discrete Cosine Transformation) represented by JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group). Besides, there are waveform coding methods such as sub-band coding, wavelet coding, and fractal coding. Further, in order to eliminate a redundant signal between images, inter-image prediction using motion compensation is carried out, and a difference signal is subjected to waveform coding.
Here, an MPEG method based on motion compensation DCT will be described. Initially, an input image of one frame to be coded is divided into plural macroblocks each having the size of 16xc3x9716 pixels. Each macroblock is further divided into four blocks each having the size of 8xc3x978 pixels, and each block of 8xc3x978 pixels is subjected to DCT and quantization. This process is called xe2x80x9cintra-frame codingxe2x80x9d.
On the other hand, using a motion detecting method such as block matching, from a frame temporarily adjacent to an object frame including an object macroblock to be quantized, a prediction macroblock having the smallest error from the object macroblock is detected, and motion compensation from the past image is carried out on the basis of the detected motion, thereby to obtain an optimum prediction block. A signal showing the motion toward the prediction macroblock having the smallest error is a motion vector. An image used as a reference for generating the prediction macroblock is called a reference image, hereinafter. Thereafter, a difference between the object block and the corresponding prediction block is obtained, and this difference is subjected to DCT to obtain a DCT coefficient. The DCT coefficient is quantized, and the quantized output is transmitted or stored together with the motion information. This process is called xe2x80x9cinter-frame codingxe2x80x9d.
The inter-frame coding has two prediction modes: prediction from a previous image in the display order, and prediction from both of previous and future images. The former is called xe2x80x9cforward predictionxe2x80x9d, and the latter is called xe2x80x9cbidirectional predictionxe2x80x9d.
On the decoder end, after restoring the quantized DCT coefficient to the original difference signal, the prediction block is obtained on the basis of the difference signal and the motion vector, and the prediction block and the difference signal are added to reproduce the image. In this conventional technique, it is premised that the size of the reference image (an image used as a reference for generating a prediction image) is equal to the size of the object image.
In recent years, plural objects constituting an image (arbitrary shape images) are separately subjected to compressive coding and transmitted, thereby to improve the coding efficiency and to enable object by object reproduction. In coding and decoding of such arbitrary shape image, the size of the image changes very often. For example, a ball becomes smaller and smaller, till at last it disappears. Further, there is a case where the size of the image (object) becomes zero.
In ordinary predictive coding, a reference image is a reproduced image just before an object image which is currently being processed. When the size of the reference image is zero, since nothing is defined in the reference image, i.e., since the reference image has no significant image content data to be used for predictive coding, predictive coding cannot be carried out. In this case, there is no conventional way except the intra-frame coding. However, generally the intra-frame coding increases the quantity of coded data and reduces the compression efficiency. When an image disappears (image size=zero) and appears frequently in a sequence of motion picture, the coding efficiency is significantly degraded. For example, in a motion picture of flashing spotlight, when the light disappears and appears in image units, all the images of lights must be subjected to the intra-frame coding.
It is an object of the present invention to provide an image predictive decoding method, an image predictive decoding apparatus, an image predictive coding method, an image predictive coding apparatus, and a data storage medium, which can realize efficient predictive coding or decoding of a variable-size image even when the size of a reference image is zero or when the reference image is completely transparent.
Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a first aspect of the present invention, there is provided an image predictive decoding method in which image data obtained by compressively coding a variable-size image using a prescribed method is input, a prediction image is generated using, as a reference image, at least one reproduced image which has been reproduced before an image being an object of decoding, and the object image is subjected to predictive decoding; wherein the prediction image is generated using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image content data to be referred to. In this method, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive decoding is carried out using another reproduced image of which size is not zero. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object to increase the compression efficiency, it is avoided that a variable-size image which has already disappeared is used as the reference image for predictive decoding. As a result, coded data obtained by efficient compressive coding that suppresses the code quantity can be appropriately decoded.
According to a second aspect of the present invention, there is provided an image predictive decoding method in which image data obtained by compressively coding a variable-size image using a prescribed method is input, a prediction image is generated using, as a reference image, a prescribed image which has been reproduced before an image being an object of decoding, and the object image is subjected to predictive decoding; wherein, when the reproduced image used as a reference image has no significant coded data to be referred to, an image having a prescribed value as its image data is used as the prediction image. In this method, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive decoding is carried out using a prediction image having a prescribed value. Therefore, in addition to the above-mentioned effects, generation of the prediction image is facilitated.
According to a third aspect of the present invention, there is provided an image predictive decoding method in which image data obtained by compressively coding a variable-size image using a prescribed method is input, a prediction image is generated using a reference image, and an image being an object of decoding is subjected to predictive decoding; wherein the prediction image is generated using, as the reference image, at least one of two reproduced signals which have been recently reproduced, which one has significant image data to be referred to. Therefore, in the case where plural objects constituting an image are subjected to compressive coding and transmitted object by object, when a prediction image is generated using forward and backward reference images, it is avoided that variable-size images which have already disappeared are used as the reference images. As a result, coded data obtained by efficient compressive coding that suppresses the code quantity can be appropriately decoded.
According to a fourth aspect of the present invention, there is provided an image predictive decoding apparatus comprising input means to which image data obtained by compressively coding a variable-size image using a prescribed method is applied; a data analyzer which analyzes the image data and outputs the image size and the image transform coefficient; a decoder which restores the image transform coefficient to an expanded difference image using a prescribed method; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using, as a reference image, the reproduced image stored in the frame memory; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator examines whether or not the reproduced image includes significant image data to be referred to, and generates a prediction image using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image data. In this apparatus, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive decoding is carried out using another reproduced image of which size is not zero. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object to increase the compression efficiency, it is avoided that a variable-size image which has already disappeared is used as the reference image for predictive decoding. As a result, coded data obtained by efficient compressive coding that suppresses the code quantity can be appropriately decoded.
According to a fifth aspect of the present invention, there is provided an image predictive decoding apparatus comprising input means to which image data obtained by compressively coding a variable-size image using a prescribed method is applied; a data analyzer which analyzes the image data and outputs the image size and the image transform coefficient; a decoder which restores the image transform coefficient to an expanded difference image using a prescribed method; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using, as a reference image, a prescribed reproduced image stored in the frame memory and corresponding to the input image; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator examines whether or not the prescribed reproduced image has significant image data to be referred to, and when the reproduced image has no significant coded data to be referred to, an image having a prescribed value as its image data is used as the prediction image. In this apparatus, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive decoding is carried out using a prediction image having a prescribed value. Therefore, in addition to the above-mentioned effects, generation of the prediction image is facilitated.
According to a sixth aspect of the present invention, there is provided an image predictive decoding apparatus comprising input means to which image data obtained by compressively coding a variable-size image using a prescribed method is applied; a data analyzer which analyzes the image data and outputs the image size and the image transform coefficient; a decoder which restores the image transform coefficient to an expanded difference image using a prescribed method; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using the reproduced image stored in the frame memory as a reference image; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator generates the prediction image using, as the reference image, at least one of two reproduced signals which have been recently reproduced, which one has significant image data to be referred to. Therefore, in the case where plural objects constituting an image are subjected to compressive coding and transmitted object by object, when a prediction image is generated using forward and backward reference images, it is avoided that variable-size images which have already disappeared are used as the reference images. As a result, coded data obtained by efficient compressive coding that suppresses the code quantity can be appropriately decoded.
According to a seventh aspect of the present invention, there is provided an image predictive coding method in which a variable-size image is input, a prediction image is generated using, as a reference image, at least one reproduced image which has been reproduced before an image being an object of coding, the object image is subtracted from the prediction image, and a difference between these images is compressively coded by a prescribed method; wherein the prediction image is generated using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image data to be referred to. In this method, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive coding is carried out using another reproduced image of which size is not zero. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object to increase the compression efficiency, it is avoided that a variable-size image which has already disappeared is used as the reference image for predictive coding, resulting in a predictive coding method capable of efficient compressive coding that suppresses the code quantity.
According to an eighth aspect of the present invention, there is provided an image predictive coding method in which a variable-size image is input, a prediction image is generated using, as a reference image, a prescribed reproduced image which has been reproduced before an image being an object of coding, the object image is subtracted from the prediction image, and a difference between these images is compressively coded by a prescribed method; wherein, when the reproduced image used as a reference image has no significant image data to be referred to, an image having a prescribed value as its image data is used as the prediction image. In this method, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive coding is carried out using a prediction image having a prescribed value. Therefore, in addition to the above-mentioned effects, generation of the prediction image is facilitated.
According to a ninth aspect of the present invention, there is provided an image predictive coding method in which a variable-size image is input, a prediction image is generated using a reference image, an object image being an object of coding is subtracted from the prediction image, and a difference between these images is compressively coded by a prescribed method; wherein the prediction image is generated using, as the reference image, at least one of two reproduced images which has been recently reproduced and includes significant image data to be referred to. Therefore, in the case where plural objects constituting an image are subjected to compressive coding and transmitted object by object, when a prediction image is generated using forward and backward reference images, it is avoided that variable-size images which have already disappeared are used as the reference images, resulting in a predictive coding method capable of efficient compressive coding that suppresses the code quantity.
According to a tenth aspect of the present invention, there is provided an image predictive coding apparatus comprising input means to which data of a variable-size image is input, which data is divided into units subjected to coding; a subtracter that obtains a difference image between an object image being an object of coding and a prediction image corresponding to the object image; a compressive encoder that converts the difference image to compressed data by a prescribed compressive coding process; a variable-length encoder that performs variable-length coding of the compressed data and outputs coded data; an expansive decoder that restores the compressed data to an expanded difference image by a prescribed expansive decoding process; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using the reproduced image stored in the frame memory as a reference image; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator examines whether or not the reproduced image has significant image data to be referred to, and generates the prediction image using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image data. In this apparatus, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive coding is carried out using another reproduced image of which size is not zero. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object to increase the compression efficiency, it is avoided that a variable-size image which has already disappeared is used as the reference image for predictive coding, resulting in a predictive coding apparatus capable of efficient compressive coding that suppresses the code quantity.
According to an eleventh aspect of the present invention, there is provided an image predictive coding apparatus comprising input means to which data of a variable-size image is input, which data is divided into units subjected to coding; a subtracter that obtains a difference image between an object image being an object of coding and a prediction image corresponding to the object image; a compressive encoder that converts the difference image to compressed data by a prescribed compressive coding process; a variable-length encoder that performs variable-length coding of the compressed data and outputs coded data; an expansive decoder that restores the compressed data to an expanded difference image by a prescribed expansive decoding process; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using the reproduced image stored in the frame memory as a reference image; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator examines whether or not the reproduced image has significant image data to be referred to and, when the reproduced image has no significant image data, an image having a prescribed value as its image data is used as the prediction image. In this apparatus, when the size of the reference image (reproduced image) is zero, i.e., when the reference image is completely transparent, predictive coding is carried out using a prediction image having a prescribed value. Therefore, in addition to the above-mentioned effects, generation of the prediction image is facilitated.
According to a twelfth aspect of the present invention, there is provided an image predictive coding apparatus comprising input means to which data of a variable-size image is input, which data is divided into units subjected to coding; a subtracter that obtains a difference image between an object image being an object of coding and a prediction image corresponding to the object image; a compressive encoder that converts the difference image to compressed data by a prescribed compressive coding process; a variable-length encoder that performs variable-length coding of the compressed data and outputs coded data; an expansive decoder that restores the compressed data to an expanded difference image by a prescribed expansive decoding process; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using the reproduced image stored in the frame memory as a reference image; and an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; wherein the prediction image generator generates the prediction image using, as the reference image, at least one of two reproduced signals which have been recently reproduced, which one has significant image data to be referred to. Therefore, in the case where plural objects constituting an image are subjected to compressive coding and transmitted object by object, when a prediction image is generated using forward and backward reference images, it is avoided that variable-size images which have already disappeared are used as the reference images, resulting in a predictive coding apparatus capable of efficient compressive coding that suppresses the code quantity.
According to a thirteenth aspect of the present invention, there is provided an image predictive coding apparatus comprising input means to which data of a variable-size image is input, which data is divided into units subjected to coding; a subtracter that obtains a difference image between an object image being an object of coding and a prediction image corresponding to the object image; a compressive encoder that converts the difference image to compressed data by a prescribed compressive coding process; a variable-length encoder that performs variable-length coding of the compressed data and outputs coded data; an expansive decoder that restores the compressed data to an expanded difference image by a prescribed expansive decoding process; a frame memory that contains a reproduced image; a prediction image generator that generates a prediction image using the reproduced image stored in the frame memory as a reference image; an adder that generates a reproduced image by adding the expanded difference image and the prediction image, and outputs the reproduced image and, simultaneously, stores the reproduced image into the frame memory; and a shape detector that detects whether the reproduced image includes significant image data to be referred to or not, on the basis of shape data showing the shape of an object and included in the variable-size image data; wherein the prediction image generator receives an output from the shape detector and, when the reproduced image has no significant image data, the prediction image generator generates the prediction image using, as a reference image, at least one reproduced image which has been recently reproduced and includes significant image data. In this apparatus, when it is detected by the shape detector that the input shape signal has a shape, the shape signal is subjected to predictive coding and, when the input shape signal has no shape, the shape signal is not subjected to predictive coding. Therefore, when plural objects constituting an image are subjected to compressive coding and transmitted object by object, it is avoided that a variable-size image which has already disappeared is used as a reference image for predictive coding, resulting in a predictive coding apparatus capable of efficient compressive coding that suppresses the code quantity.
According to a fourteenth aspect of the present invention, there is provided a data storage medium that contains a program for implementing a predictive decoding process by a computer, wherein the program is constructed so that the computer executes an image predictive decoding process according to any of the above-described image predictive decoding apparatuses. Therefore, it is possible to realize, by software, a predictive decoding process that can decode coded data obtained by efficient compressive coding that suppresses the code quantity.
According to a fifteenth aspect of the present invention, there is provided a data storage medium that contains a program for implementing a predictive coding process by a computer, wherein the program is constructed so that the computer executes an image predictive coding process according to any of the above-described image predictive coding apparatuses. Therefore, it is possible to realize, by software, a predictive coding process capable of efficient compressive coding that suppresses the code quantity.
According to a sixteenth aspect of the present invention, there is provided a data storage medium that contains a program for implementing a predictive coding process by a computer, wherein the program is constructed so that the computer executes an image predictive coding process according to any of the above-described image predictive coding apparatuses. Therefore, it is possible to realize, by software, a predictive coding process capable of efficient compressive coding that suppresses the code quantity.